Error Estimates for Approximate Optimization by the Extended Ritz Method

An alternative to the classical Ritz method of approximate optimization is investigated. In the extended Ritz method, sets of admissible solutions are approximated by their intersections with linear combinations of n-tuples from a given set. This approximation scheme, called variable-basis approximation, includes functions computable by trigonometric polynomials with free frequencies, neural networks, free-node splines, and many other nonlinear approximating families. Estimates of rates of approximate optimization by the extended Ritz method are derived. For problems with argminima, upper bounds on rates of convergence of approximate infima and argminima to a global infimum and argminimum are expressed in terms of the “degree” n of variable-basis functions, of the modulus of continuity of the functional to be minimized, of the modulus of Tychonov wellposedness of the problem, and of certain norms tailored to the type of variable basis. Classes of high-dimensional optimization problems are described, for which rates of approximate optimization do not exhibit the curse of dimensionality with respect to the number of variables of admissible solutions. The results are applied to convex best approximation problems and kernel methods in machine learning.